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The 2 -year impacts of 11 welf are-to-work programs across the 
United States were evaluated to determine the relative effectiveness of 
different approaches to moving individuals from welfare to work. All 11 
programs operated under the Job Opportunities and Basic Skills Program. Four 
programs were employment focused (three low-enforcement and one 
high-enforcement programs) , and seven were education focused (five 
high-enforcement and two low-enforcement programs) . Although the 
employment -focused programs produced larger gains in employment and earnings 
over the 2-year follow-up period, concern was expressed that their effects 
may not be sustained over the long term. Several education- focused programs 
began to show moderate impacts in year 2. All programs reduced welfare 
dependency to some degree. The programs did not have widespread, large, or 
consistent effects on sample members' children; however, positive and 
negative effects occurred in some programs. High-enforcement programs did not 
produce the largest impacts, but low-enforcement programs resulted in only 
small effects. (Ninety tables/figures are included. The following items are 
appended: supplementary tables and figures; information about family 
circumstances, child outcomes, and impacts for families with preschool -age 
children; a survey response analysis; and a comparison of impacts estimated 
from survey and unemployment insurance data. The bibliography lists 55 
references . ) (MN) 
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Brown, June Cook, Cathy Craner, Oren Cyphers, Hilda Davis, Bob Earnest, Carol Eckel, Doro- 
thy Fuller Angel Grogen, Veda Latin, Bruce Lowry, Linda Montgomery, Ann Pickar, Kei 
Quitevis, Will Reinhart, Pam Ruddell, C. L. Thames, Roger Zwemke, Jodi Davich, Mardica 
Hicks, Maureen Judge-Morris, Nan Poppe, and Julie Wyckoff-Byers. 
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Executive Summary 



The Personal Responsibility and Work Opportunity Reconciliation Act of August 19961 
ended the Aid to Families with Dependent Children (AFDC) program, one of the nation’s princi- 
pal safety nets for poor families. Among its provisions, the law replaced AFDC with a block 
grant program, Temporary Assistance for Needy Families (TANF), and created financial incen- 
tives for states to run mandatory, work-focused welfare-to-work programs. While these types of 
programs are not new, various aspects of the 1996 law increase their importance: federal funds 
now may not be used to support most families on welfare for longer than five years and a number 
of states and localities have shorter welfare time limits; states face financial penalties if they fail 
to meet TANF-defined “participation standards,” which require large proportions of welfare re- 
cipients to be in work or work-related activities; and states must have a plan for how they will 
require recipients to work after two years of assistance. 

To meet the new challenges of the federal welfare legislation, state and local administra- 
tors and policy makers need to know about the types of welfare-to-work program approaches that 
can quickly move substantial numbers of people into work and off welfare. This report provides 
such guidance, by analyzing the effectiveness of 11 mandatory welfare-to-work programs oper- 
ated in seven locales. The sites included in the evaluation are Atlanta, Georgia; Columbus, Ohio; 
Detroit and Grand Rapids, Michigan; Oklahoma City, Oklahoma; Portland, Oregon; and River- 
side, California. 

The report is one in a series from an evaluation of the programs called the National 
Evaluation of Wei far e-to- Work Strategies (NEWWS), conducted by the Manpower Demonstra- 
tion Research Corporation (MDRC) under contract to the U.S. Department of Health and Human 
Services (HHS), with support from the U.S. Department of Education. Child Trends, as a 
subcontractor, is conducting the analyses of outcomes for young children (the Child Outcomes 
Study). Two other recent reports (both also published in 2000 by the U.S. Department of Health 
and Human Services, Administration for Children and Families and Office of the Assistant Sec- 
retary for Planning and Evaluation, and the U.S. Department of Education) should be viewed as 
“companion” documents to this report: Impacts on Young Children and Their Families Two 
Years After Enrollment: Findings from the Child Outcomes Study, prepared by Sharon M. 
McGroder, Martha J. Zaslow, Kristin A. Moore, and Suzanne M. LeMenestrel, Child Trends; and 
Do Mandatory Welfare-to-Work Programs Affect the Well-Being of Children? A Synthesis of 
Child Research Conducted as Part of the National Evaluation of Welfare-to-Work Strategies, 
prepared by Gayle Hamilton, MDRC, with Stephen Freedman, MDRC, and Sharon M. 
McGroder, Child Trends. 

Each of the 1 1 studied programs operated under the federal Job Opportunities and Basic 
Skills Training (JOBS) program, which preceded TANF. Unlike TANF, these programs did not 
impose a time limit on eligibility for welfare assistance. However, they shared TANF s primary 
goal of moving welfare recipients into paid work and off assistance. Further, among the 1 1 pro- 
grams some are strongly employment-focused, the welfare-to-work strategy favored under 
TANF, and some are strongly basic education-focused, an approach possible under TANF but 
more prevalent during the late 1980s and early 1990s. (Overall, the present results pertain to the 
period between 1991 and 1996.) The programs varied in other ways, including how broadly the 
participation mandate was applied to the welfare caseload and how strictly it was enforced, the 
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amount of child care support provided for program participation or employment, and methods of 
e ty o n abo r niark et ^ pr0gramS als0 Served different welfare populations and operated in a vari- 



Taking advantage of the array of programs studied as part of the evaluation this report 
a dresses the following critical question: What works best, and for whom? The report distin- 
guishes between employment-focused and basic education-focused programs, as well as between 
eve s o enforcement of the participation mandate. Taking into account these two dimensions of 
program characteristics, plus the types of program activities to which welfare recipients were as- 
signed, four categories of welfare-to-work program approaches emerge: 



• employment-focused programs, with first assignments made to job search and 
a high level of participation mandate enforcement; 

• employment-focused programs, with first assignments made to job search, ba- 
sic education, or vocational skills training and a high level of participation en- 
forcement (only one program falls into this category); 

• education-focused programs, with first assignment made to basic education or 
skills training and a high level of participation enforcement; and 

• education-focused programs, with first assignments made to basic education 
or skills training and a low level of participation enforcement. 

Exhibit ES-1 categorizes the sites’ programs. Notably, four of the sites (Atlanta, Grand 
apids, Riverside, and Columbus) operated two different programs simultaneously, to enable 
rigorous side-by-side tests of the comparative effectiveness of various approaches. Three sites 
implemented a Labor Force Attachment (LFA) program as well as a Human Capital Develop- 
ment (HCD) program, versions of employment-focused and education-focused programs that 
magnified the differences between the two types of approaches. The fourth site, Columbus im- 
plemented a program using a traditional (TRD) case management model, in which welfare eligi- 
bility and employment program functions were performed by separate staff members and a pro- 
gram using an Integrated (INT) case management model, in which these two functions were 
performed by the same staff member. These eight programs in four sites, described in more detail 

refc "'^ tooughout by their site name and shortened program model name 
HCD, TRD, or INT). 



• , i • ? 1S ™P ortant t0 note that the studies of the programs in the education-focused category 
yield information about the effects of increasing welfare recipients’ participation in basic educa- 
ton programs (including Adult Basic Education, GED preparation, and English as a Second 
Language classes) and, to a much lesser extent, in vocation skills training programs, but not in 
college. On their own, many welfare recipients enroll in various types of education or training 
classes and reap benefits from them; the education-focused programs in the evaluation, however 
sought to increase participation in education or training activities beyond what would normally 
occur. As will be discussed below, most of the programs did indeed increase such participation 
but the increases in enrollments were in basic education courses and, to some degree in voca- 
tional training courses, and not in college-level ones. 
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National Evaluation of Welfare-to-Work Strategies 
Exhibit ES-1 

Categorizing NEWWS Programs, by Approach, First Activity, and Enforcement Level 



Employment-locused approach 


Education-tocused approach 


Job search first 


Varied first activity 


Education or i 


training first 


High Enforcement 


High Enforcement 


High Enforcement 


Low Enforcement 


Atlanta LFA 
Grand Rapids LFA 
Riverside LFA 


Portland 


Atlanta HCD 
Grand Rapids HCD 
Riverside HCD 
Columbus Integrated 
Columbus Traditional 


Detroit 

Oklahoma City 



This report analyzes the programs’ effects for single-parent welfare recipients, focusing 
on results for the two years after individuals entered the programs. This is an important period in 
which to gauge whether programs moved recipients from welfare to work. Many states and lo- 
calities now terminate welfare eligibility after two years. In addition, prior research has shown 
that many individuals on welfare for at least two years will likely remain on the rolls for a con- 
siderably longer time. Under TANF, these individuals would be in jeopardy of reaching their 
five-year limit on federal funding for welfare benefits. Consequently, the two-year results for 
these 1 1 programs will become a benchmark for the next generation of welfare initiatives. 

The report explores the following questions: 

• Which welfare-to-work program approaches were most successful in helping 
welfare recipients to receive the program services or attain the skills or 
credentials that could enhance their chances of finding employment? 

• Which approaches were most successful in helping welfare recipients to find 
paid work and leave welfare within the two-year follow-up period and to re- 
main off welfare? Did any approaches help individuals to get a “good” job, 
that is, a full-time job with health benefits? 

• Which approaches were most successful in increasing welfare recipients in- 
come and helping them move out of poverty? 

• Did any approaches positively or negatively affect the well-being of children? 

• Which approaches were most successful in achieving self-sufficiency for those 
who were at high risk for long stays on welfare? 

The NEWWS Evaluation uses an unusually strong research design, a random assignment 
experiment, to estimate program effects. In each site individuals who were required to participate 
in the program were assigned at random to either a program group (in some sites, one of two 




* * 



ES-3 



17 



program groups) or a control group. Program group members had access to program-provided 
services and were required to participate in the program or risk a reduction in their monthly wel- 
fare grant. Control group members received no mandatory welfare-to-work program services but 
could seek similar services on their own in the community. This random assignment design as- 
sures that within each site there are no systematic differences between the background character- 
istics of program and control group members when they enter the study. In addition, within each 
site program and control group members are subject to the same welfare grant levels, labor mar- 
ket conditions, and other environmental factors. As a result, any subsequent differences in out- 
comes between the groups within each site can be attributed with confidence to the effects of the 
program. These differences, called impacts, can then be compared across sites, yielding a much 
more accurate determination of which types of programs are high and low performers than sim- 
ple comparisons of statistics, such as welfare caseload reductions, across localities or states. 



I. Findings in Brief 

An examination of the range of effects achieved by all 1 1 programs yielded the following 
information about which welfare-to-work program strategies are more or less successful in help- 
ing welfare recipients achieve self-sufficiency: 

All programs, regardless of their approach, increased participation in activities de- 
signed to promote employment during the two-year follow-up period. As expected, employ- 
ment-focused programs increased participation primarily in job search activities, whereas educa- 
tion-focused programs raised participation levels primarily in basic education and vocational 
skills training classes. Very different patterns of participation impacts were found for individuals 
who entered the study with a high school diploma or GED certificate and for those who did not 
have these credentials. In most education-focused programs participation impacts were concen- 
trated among those without a high school diploma or GED and resulted primarily from large in- 
creases in attendance in basic education; only small increases in attendance in post-secondary 
education or vocational training were found for the education-focused programs, and they were 
generally among only high school graduates or GED holders. In contrast, large impacts on par- 
ticipation in job search were achieved for both groups in the employment-focused programs. 

Some education-focused programs, as well as the Portland program, were able to 
produce relatively large impacts (about 10 percentage points) on GED attainment among 
sample members who did not have a high school diploma or GED certificate at study entry. 

Of the seven education-focused programs, Grand Rapids HCD, Riverside HCD, and Columbus 
Traditional programs had this effect. Portland’s program, in addition to boosting GED receipt, 
increased the rate at which those without education credentials obtained a trade license or certifi- 
cate by 12 percentage points. For sample members with a high school diploma or GED certificate 
at study entry, only three programs (Atlanta LFA and HCD and Grand Rapids HCD) increased 
receipt of a trade license or certificate. 

As expected, employment-focused programs produced larger gains in employment 
and earnings over the two-year follow-up period than education-focused programs but 
these effects may not be sustained everywhere in the long run. Except in Riverside the site 
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with the most difficult labor market, a majority of control group members found jobs on their 
own initiative at some point within two years of random assignment and, as a group (including 
zeroes for noneamers), had average earnings during the second year of follow-up ranging from 
$2,127 (Oklahoma City) to $3,978 (Columbus). In Portland program group members attained the 
largest earnings increase among all programs, averaging more than $900 per year in earnings 
above control group members. Equally important, employment and earnings gains in Portland 
grew larger over time and reached their highest levels at the end of year 2, the end of the short- 
term follow-up period available for this report. The other employment-focused programs pro- 
duced moderate earnings increases, ranging from $400 to $650 per year, that grew smaller to- 
ward the end of year 2. 

Several of the education-focused programs began to show moderate impacts in year 

2. By the end of year 2 all but two of the education-focused programs had attained increases in 
employment and earnings that equaled or exceeded the gains achieved by all employment- 
focused programs except Portland’s. The two exceptions to this pattern, the Riverside HCD and 
Oklahoma City programs, did not raise employment or earnings levels in year 2. Overall, these 
results underscore the importance of tracking the effects of education-focused programs over a 
longer term. 

All programs reduced welfare dependency to some degree. Control group members in 
all but one site remained on welfare for an average of 16 to 20 months during the two-year fol- 
low-up period and received payments averaging between $3,624 (Oklahoma City) and $10,302 
(Riverside HCD) during this same period. Seven of the 1 1 programs, a mixture of employment- 
and education-focused approaches, decreased cumulative welfare expenditures by more than 10 
percent, a historically large effect; welfare reductions in the other four programs were smaller. 
Portland’s program produced a large decrease in welfare receipt that persisted at a high level 
throughout the follow-up period, showing a 12 percentage point decrease in welfare receipt dur- 
ing the last quarter of the two-year period; all other programs had reduced welfare receipt at this 
point by 3 to 7 percentage points. All in all, however, at least 40 percent of sample members in 
the programs were still relying to some extent on welfare at the end of two years. 

Most programs increased sample members’ reliance on earnings, as opposed to wel- 
fare, but family net incomes were largely unchanged. As a result, the programs lifted few 
families above the poverty line. Reductions in welfare, Food Stamps, and other benefits gener- 
ally matched or exceeded earnings gains. Including estimates of the Earned Income Tax Credit 
(EITC) as income produced little change in this finding for all programs except Portland’s, which 
attained the largest and most consistent gain in total income ($238, or $425 including the EITC 
estimate, for year 2 of the follow-up) and also produced a small increase in the proportion with 
incomes above the poverty level (4 percentage points, or 7 percentage points including the EITC 
estimate, in year 2). 

Although no programs had pervasive negative effects on sample members, some in- 
dividuals were adversely affected. In year 2 of follow-up six programs (some employment- 
focused and some education-focused) produced small increases in the proportion of sample 
members with combined income from AFDC, Food Stamps, and earnings equivalent to less than 
50 percent of poverty levels. In addition, several programs (representing both types of ap- 
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proaches) increased the rate at which individuals left welfare without a job. Finally, some pro- 
grams that increased employment also decreased family health insurance coverage (as reported 
by parents) and increased out-of-pocket child care expenditures. 

The programs did not have widespread, large, or consistent effects on the children 
of sample members, but positive and negative effects occurred in some programs. No pro- 
grams in the evaluation provided direct services (with the exception of child care assistance) to 
children. Program-produced changes in the lives of sample members (virtually all mothers) may, 
nevertheless, influence the well-being of children. There is evidence that some of the programs 
affected the likelihood of at least one child in a family having behavioral, educational, or health 
and safety problems. There was not, however, a consistent pattern of benefit or harm to children. 
In addition, employment- and education-focused programs did not appear to affect children dif- 
ferently; there was no consistent evidence that one particular approach affected children more or 
less or was more likely to help or harm children. 

Several employment- and education-focused programs attained at least moderate 
employment and earnings gains for the “most disadvantaged” sample members. Five pro- 
grams (Portland, Grand Rapids LFA and HCD, and Riverside LFA and HCD) increased em- 
ployment and earnings for individuals who at study entry did not have a high school diploma or 
GED, had not worked in the prior year, and had been on welfare cumulatively for two years or 
more. These five programs and two others (Detroit and Columbus Integrated) also reduced the 

amount of time that the most disadvantaged individuals spent on welfare during the follow-up 
period. 



High enforcement programs did not produce the largest impacts, but low enforce- 
ment programs resulted in only small effects. Programs in which staff closely monitored indi- 
viduals’ attendance in program activities, followed up quickly when problems arose, and swiftly 
imposed financial sanctions when individuals did not comply with program requirements, were 
present among both the employment- and education-focused programs. High enforcement pro- 
grams, notably those in Grand Rapids and Columbus, did not necessarily produce the largest im- 
pacts. However, the two low enforcement programs — Oklahoma City and, in its early stages, 
Detroit — yielded only small impacts. It thus appears that a minimum level of enforcement by 
program staff is required to produce at least moderate earnings and welfare impacts, presumably 
because this extra “push” is needed in order to engage in program activities those who normally 
would not participate on their own initiative. 



While many programs achieved positive effects on employment, earnings, and re- 
duced use of welfare, few achieved large effects, except for Portland. The Portland program 
was^unusually successful in substantially increasing employment and earnings, helping people to 
get “good” jobs, lowering welfare receipt, and achieving these outcomes for a cross section of 
sample members. The results are probably due to a combination of factors. While its employment 
message was strong, the program offered high-quality education and training services as well as 
job search, enforced a participation mandate, and had strong job development and placement ser- 
vices. In addition, contextual factors may have contributed to the program’s success. In particu- 
lar, it worked with a less disadvantaged welfare caseload (relative to the other studied programs) 
and operated within a good labor market with a relatively high state minimum wage. 
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The remainder of this summary details these findings. First,' however, it describes the key 
welfare-to-work program approaches contrasted in the analysis and explains the evaluation’s re- 
search design and samples. 

II. Program Approaches and Implementation Features 

As noted above, the evaluation’s sites implemented very different programs; in fact, the 
research designs in several of the , sites were set up to rigorously compare the effects of specific 
program approaches. This section discusses the two key implementation features used in this re- 
port to define four broad program approaches. In addition, for context, other major program di- 
mensions are described. 

A. Employment- or Education-Focused 

Since the late 1960 s, welfare-to-work programs seeking to increase welfare recipients’ 
self-sufficiency have emphasized one of two strategies. One strategy emphasizes quick employ- 
ment, reflecting the belief that individuals can best build their employability, and eventually 
achieve self-sufficiency, through actual work, even if their initial jobs are minimum wage and 
without fringe benefits: The other strategy emphasizes skill-building, particularly in the educa- 
tion area, reflecting the view that individuals should first invest in education or training to enable 
them to eventually obtain higher-wage, longer-lasting jobs with health insurance coverage. The 
1 1 NEWWS programs blend elements of both strategies to varying degrees. 

As shown in Exhibit ES-1 four programs (Atlanta, Grand Rapids, and Riverside LFA and 
Portland) were “employment focused.” They provided job search assistance to a large segment of 
their caseload and encouraged enrollees to find work as quickly as possible. Further, both the 
Portland and Riverside programs employed full-time job developers to help place program enrol- 
lees in unsubsidized jobs. 

The three LFA programs, however, differed from Portland’s program in important ways. 
The LFA programs routinely assigned individuals to job search assistance, usually job club, as 
their first activity, whereas Portland’s program offered GED preparation classes to those deamed 
by case managers to have a good chance of attaining a GED certificate relatively quickly. (Ac- 
tivities initially assigned are an important clue to the “treatment” experienced by welfare recipi- 
ents, as many people leave welfare or become exempt or temporarily excused from welfare-to- 
work programs prior to being assigned to a second program activity.) Further, Portland case 
managers encouraged enrollees to hold out for jobs that paid well above the minimum wage and 
offered the best chance for long-lasting and stable employment. In contrast, case managers in the 
LFA programs, especially in Riverside, stressed the value, of starting off with any job, even a 
low-paying one, and then advancing toward more stable and better-paying jobs in the future. 

Seven programs (Atlanta, Grand Rapids, and Riverside HCD; Columbus Integrated and 
Traditional; and Detroit and Oklahoma City) can be characterized as “education-focused.” (See 
Exhibit ES-1.) A large percentage oif enrollees in these programs were initially assigned to some 
type of skill-building activity. The types of activities to which enrollees were first assigned de- 
pended, in part, on the level of educational attainment that individuals had achieved prior to en- 



tenng the program. Those who had not completed high school or received a GED certificate but 
who were assessed by case managers as having high school-level skills were assigned to GED 
preparation classes. Those with lower reading or math levels were assigned to Adult Basic Edu- 
cation classes. In addition, non-English speakers could be assigned to English as a Second Lan- 
guage (ESL) programs. Finally, those who had completed high school or held a GED certificate 
could be assigned to vocational training or employment-oriented skills courses at local commu- 
nity colleges. All in all, however, assignments to GED preparation or basic education courses 
were more common than assignment to vocational training programs in these education-focused 
programs, primarily as a result of welfare recipients’ low levels of educational achievement; en- 
rollment in college played an even smaller role. 

Some differences existed among the seven education-focused programs. The three HCD 
programs usually assigned enrollees to education or training programs as their first activity. Case 
managers in Columbus, Detroit, and Oklahoma had more discretion over activity assignments, 
but, in practice, most program enrollees were initially assigned to education or training activities 
in these sites as well. Riverside’s HCD program was also unique among this group in that it did 
not serve high school graduates and GED holders who, at program entry, scored above minimum 
levels in reading and math tests. 

B - High or Low Enforcement of the Participation Mandate 

The degree to which a program enforces a participation mandate can be viewed as a 
product of three factors: how wide a cross section of the welfare caseload is enrolled in a pro- 
gram; how closely a program monitors individuals’ participation; and how swiftly and consis- 
tently a program imposes financial sanctions, that is, reductions in monthly welfare grants, on 
those who do not participate. 

All four employment-focused programs, and five of the seven education-focused pro- 
grams, can be considered high enforcement programs; the remaining two education-focused pro- 
grams, Detroit and Oklahoma City, can be considered low enforcement programs. While techni- 
cally requiring enrollment from a cross section of their “mandatory” caseloads, these latter two 
programs put a priority on working with those individuals who expressed interest in participating 
in the program. In addition, resource constraints kept staff in these sites from closely monitoring 
individuals participation in program activities. Finally, staff in these two sites rarely invoked 
financial sanctions. In contrast, program staff in the other programs generally enrolled and 
worked with a cross section of the welfare applicants and recipients who were required to par- 
ticipate; monitored participation more closely; and, especially in Columbus and Grand Rapids, 
frequently invoked sanctions for nonparticipation. 

C. Other Key Program Features 

Other implementation features, beyond those discussed above, can also potentially influ- 
ence a program’s effectiveness. Two of them — the level of child care support provided and the 
structure of program case management — are described here. 

All 1 1 studied programs offered child care assistance to welfare recipients who needed it 
while they were participating in program activities or employed. Oklahoma City, Portland, and 




ES-8 



22 



•> . i 



Detroit provided the strongest staff support for arranging child care. Staff in these programs 
helped to make child care arrangements and also helped those who found jobs to obtain transi- 
tional child care assistance. In contrast, case managers for both Riverside programs did not pro- 
vide much assistance in setting up child care arrangements, encouraged enrollees to use low- or 
zero-cost informal child care while they were participating in program activities, and did not ac- 
tively promote the use of transitional child care benefits. 

The programs also differed in their case management strategies. Columbus Integrated, 
Portland, and Oklahoma City implemented an “integrated case management” staffing arrange- 
ment. That is, case managers in these sites combined responsibilities normally performed by in- 
come maintenance staff (determining welfare eligibility, calculating welfare grants, invoking fi- 
nancial penalties, and arranging for transitional benefits) with responsibilities usually assigned to 
wel far e-to- work program staff (assigning enrollees to employment-related activities, arranging 
for child care, and monitoring participation). Columbus Integrated and Portland staff had suffi- 
cient resources and small enough caseloads to perform both of these roles, enabling them to pro- 
mote a consistent self-sufficiency message. In contrast, in Oklahoma City limited resources and 
large caseloads led case managers to put most of their overall emphasis on the financial functions 
of their job. 

The Atlanta, Grand Rapids, and Riverside LFA and HCD, Columbus Traditional, and De- 
troit programs all used a traditional case management structure, in which each welfare recipient 
had two different case managers. Commonly, income maintenance workers knew little about the 
welfare-to-work program in their site. Among these sites, the staffing division was most pro- 
nounced in Detroit. 



III. Research Designs and Samples 

In Atlanta, Grand Rapids, and Riverside welfare recipients were randomly assigned to 
either an LFA or an HCD program group or to a control group. (See Exhibit ES-2.) Both types of 
programs operated simultaneously in these three sites. In Columbus a three-group random as- 
signment design was used as well. Here, the two program groups represented two case manage- 
ment models: integrated and traditional. The remaining three sites in the evaluation — Oklahoma 
City, Detroit, and Portland — used random assignment to test the effectiveness of established 
programs, as opposed to programs designed to meet research protocols; individuals were ran- 
domly placed in either a group that entered the program or a nonprogram control group. Note 
that control group members were eligible for child care assistance similar to that offered to pro- 
gram group members if they were participating in nonprogram activities in which they had en- 
rolled on their own. 

Individuals were randomly assigned to research groups over approximately a two-year 
period in each site. Random assignment for the evaluation began in June 1991 in Riverside and 
ended in December 1994 in Portland. Thus, the results presented in this report cover the calendar 
period from June 1991 (the first sample member’s entry into the study) through December 1996 
(the last month of the two-year follow-up for the last sample member randomly assigned in Port- 
land). 
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Differences in research design and random assignment procedures affected the composi- 
tion, and thus comparability, of the samples across sites. (See Exhibit ES-2.) In five of the seven 
sites AFDC applicants and recipients were randomly assigned while attending a program orienta- 
tion; in the other two sites (Columbus and Oklahoma City) individuals were randomly assigned 
before they were referred to a program. Since some individuals typically exit welfare for em- 
ployment or other reasons before attending a program orientation, the samples in Columbus and 
Oklahoma City include a larger share of individuals who quickly left welfare. 2 

The programs also differed in how broadly or narrowly they targeted enrollment. Most 
notably, Oklahoma City randomly assigned only welfare applicants (that is, persons in the proc- 
ess of applying for welfare), including those whose application for assistance was not yet ap- 
proved. Additionally, Detroit, Grand Rapids, Oklahoma City, and Portland extended their pro- 
gram coverage to mothers with children as young as age 1, whereas the remaining programs 
exempted parents whose youngest child was under age 3. Riverside limited enrollment in its 
HCD program to individuals determined by program regulations to need basic education because 
they lacked a high school diploma or GED certificate, attained low scores on a reading or math 
exam administered at program entry, or had limited proficiency in English. Finally, other pro- 
grams limited enrollment (and thus those eligible for random assignment) by capping caseloads 
for program staff and establishing waiting lists for enrollees (Atlanta) or by excluding those who, 
in the judgment of program staff, had serious barriers to participation (Portland). 

Because of these and other factors, the research samples differed across the seven sites in 
key background characteristics likely to affect individuals’ chances of finding employment and 
leaving welfare. For instance, excluding the Riverside FICD program, the proportion of sample 
members who had completed high school or attained a GED certificate prior to random assign- 
ment ranged from 55 percent (Oklahoma City) to 66 percent (Portland); the proportion who had 
ever worked full time for at least six months for the same employer ranged from 43 percent (Co- 
lumbus) to 77 percent (Portland); and, excluding Oklahoma City, between 28 and 50 percent of 
sample members in the sites had received welfare cumulatively for five years or more. 



IV. Findings 

A. Program Participation and Enforcement 

• Many control group members took part in education and training activi- 
ties on their own initiative. AH programs, however, were able to increase 
participation levels in employment-related activities above the control 
groups’ rate of activity during the two-year follow-up. The size of the in- 
crease was associated with the degree of enforcement of the participation 
mandate, but not with the program approach. 



Riverside, Grand Rapids, and Portland implemented an additional random assignment study of the effects 

independent of participation in welfare-to-work program activities — of referring AFDC applicants and recipients 
to a welfare-to-work program. Random assignment for this study took place at income maintenance offices. The 
results of this supplemental study are not included in this report. 
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Between 1 9 and 42 percent of control group members surveyed in each site reported par- 
ticipating during the two-year follow-up period in an employment-related activity, such as basic 
education, skills training, post-secondary education, or formal job search. As shown in Exhibit 
ES-3, all programs increased participation beyond these levels of self-initiated activity, from 9 to 
40 percentage points above control group participation levels. Overall, program participants were 
generally involved in activities for at least several months. 

All but one of the programs with high enforcement of the participation mandate (includ- 
ing both employment- and education-focused programs) produced large impacts on participation 
(above 20 percentage points). Participation impacts were much smaller for the two low enforce- 
ment programs (Detroit and Oklahoma City). In these two sites the programs’ efforts increased 
the number of welfare recipients who participated in activities only slightly beyond what they 
would have done on their own, in the absence of a mandatory welfare-to-work program. 

• As expected, all of the employment-focused programs produced large in- 
creases in participation in job search activities. Some also produced small 
increases in participation in education and training. 
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Exhibit ES-3 

Impacts on Participation in Any Employment-Related Activity 



45 

40 



35 • • 

Atlanta LFA: 



29 

30 Portland: 




Employment-Focused 

Job Search First Varied First Activity 

High Enforcement High Enforcement 



Riverside HCD: 
40 




Education or Training First Education or Training First 
High Enforcement Low Enforcement 



SOURCE: MDRC calculations from the Two-Year Client Survey. 
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The four employment-focused programs increased job search participation by 27 (Grand 
Rapids LFA) to 32 percentage points (Portland and Riverside LFA), compared with control group 
levels. (See Exhibit ES-4.) The programs achieved large gains for people who entered the program 
with a high school diploma or GED certificate and for nongraduates. Enrollees in the employment- 
focused programs could be assigned to short-term education or training if they completed job 
search without finding employment (or, in Portland, at program entry). The Atlanta LFA and Port- 
land programs produce small increases in participation in education or training. 

• Most of the education-focused programs raised participation levels in 
education or training. To a lesser extent, the programs also increased par- 
ticipation in job search. 

As shown in Exhibit ES-4, the education-focused programs increased participation in 
education or training by 10 to 35 percentage points (Oklahoma and Riverside HCD, respectively) 
compared with control group levels. (Detroit’s increase in education or training participation was 
not statistically significant.) 

While the increases for some programs were small when all sample members are consid- 
ered, most of the education-focused programs achieved large increases in participation in educa- 
tion or training for sample members lacking a high school diploma or GED certificate at random 
assignment (not shown in Exhibit ES-4). Most of these increases are accounted for by participa- 
tion in basic education. 

When enrollees in the education-focused programs completed education or training, they 
were often assigned to job search. As Exhibit ES-4 illustrates, all of the education-focused pro- 
grams raised job search participation levels to some extent; impacts were similar for high school 
graduates and nongraduates. 

• Most programs produced only small increases in participation in work 
experience or on-the-job training. 

TANF participation requirements encourage states to enroll welfare recipients in unpaid 
work or on-the-job training. None of the programs in the evaluation made extensive use of these 
activities, but most were able to produce small impacts on participation in such activities because 
even fewer control group members participated in them. (Participation impacts on these activities 
are not shown in Exhibit ES-4.) 

• The 11 programs varied widely in their use of financial sanctions, or 
AFDC grant reductions, to enforce mandatory participation require- 
ments. Sanction rates in most of the employment-focused programs were 
moderate, but rates in the education-focused programs ranged from very 
low to very high. 
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Exhibit ES-4 



Impacts on Participation in Job Search and Education or Training 





□ Job Search 
■ Education or Training 



Education-Focused 




Job Search Firsl Varied First Activity 

High Enforcement High Enforcement 



Education or Training First 
High Enforcement 



Education or Training First 
Low Enforcement 



KEY: ATL = Atlanta GR * Grand Rapids R1V « Riverside POR - Portland COL * Columbus DET- Detroit OKC - Oklahoma City 



SOURCE: MDRC calculations from the Two-Year Client Survey. 

In three of the four employment-focused programs (Atlanta and Riverside LFA and Port- 
land) between 1 1 and 1 8 percent of program group members reported that they were sanctioned 
for noncompliance with program participation requirements during the follow-up period. Three 
education-focused programs (Grand Rapids HCD and Columbus Integrated and Traditional) and 
an employment-focused program (Grand Rapids LFA) had high sanction rates, ranging from 26 
to 32 percent of program group members. At the other extreme, almost no program group mem- 
ber in the low enforcement education-focused programs in Detroit and Oklahoma City reported 
being sanctioned. 

B. Receipt of Education or Training Credentials 

• Some of the education-focused programs, as well as the Portland pro- 
gram, produced relatively large impacts on GED certificate attainment 
among sample members who entered the program without a high school 
diploma or GED certificate. 

As noted above, most education-focused programs increased participation in basic educa- 
tion among nongraduates, but only three of these programs (Grand Rapids and Riverside HCD 
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and Columbus Traditional) increased GED certificate attainment for this subgroup. Impacts on 
GED receipt ranged from 8 to 1 1 percentage points. Notably, Portland achieved similar gains in 
GED receipt. (The other three employment-focused programs had no effect on GED attainment.) 

• For those entering with a high school diploma or GED, a few programs 
increased the proportion who received a trade license or certificate. One 
program increased the proportion of nongraduates who received a trade 
credential. 

Two education-focused programs (Atlanta and Grand Rapids HCD) and one employ- 
ment-focused program (Atlanta LFA) increased receipt of a trade license or certificate for sample 
members in the graduate subgroup. Impacts ranged from 5 percentage points (Atlanta LFA) to 1 1 
percentage points (Atlanta HCD). Portland increased receipt of a trade license or certificate by 12 
percentage points among those entering the program without a high school diploma or GED. 
(Only Portland’s program had this effect for the nongraduate subgroup.) 

C. Employment and Earnings 

• Employment-focused programs produced larger gains in employment 
over the two-year follow-up period than most of the education-focused 
programs. 

Six of the seven sites in the evaluation experienced economic growth and strong labor 
markets during the first years of follow-up; aided by these conditions, a majority of control group 
members in these sites (from 58 to 72 percent) worked for pay at some point during the two-year 
follow-up period. Jobs were much harder to find in Riverside; only 45 percent of control group 
members were employed during the follow-up period. 

As shown in Exhibit ES-5, all four employment-focused programs increased two-year 
employment levels, from 5 percentage points (Atlanta LFA) to 15 percentage points (Riverside 
LFA). (Exhibit ES-5 shows outcomes for both program and control groups and the differences 
between the two groups’ outcomes, that is, the impacts; other exhibits present only the impacts 
for the various outcomes discussed.) As described above, education-focused programs delayed 
job finding in the short term. Not surprisingly, employment gains for most of these programs fell 
below those of the employment-focused programs. Three of the seven education-focused pro- 
grams produced no statistically significant increase in employment (Columbus Integrated and 
Traditional and Oklahoma), and the other education-focused programs increased employment 
between 3 and 9 percentage points (Atlanta and Riverside HCD, respectively). 

• Employment-focused programs produced much larger gains in earnings 
over the two-year follow-up period than education-focused programs. 

Earnings for control group members in the seven sites averaged between $3,133 and 
$6,892 (including zeroes for those with no earnings) over the two-year follow-up period. As Ex- 
hibit ES-6 illustrates, Portland increased earnings by an average of $1,842 per program group 
member. This earnings gain is much larger than that of the other three employment-focused pro- 
grams and exceeds that of all previously evaluated mandatory welfare-to-work initiatives, except 
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Exhibit ES-5 

Program Impacts on Selected Measures of 
Earnings, Employment, and AFDC Payments and Receipt 



— - Sample Program Control Difference Percentage 

Site and Program Size Group Group (Impact) Change (%) 



Atlanta Labor Force Attachment 3,833 

Grand Rapids Labor Force Attachment 3,012 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Portland 5,547 

Atlanta Human Capital Development 3,881 

Grand Rapids Human Capital Development 2,997 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4,459 

Oklahoma City 8,677 

Atlanta Labor Force Attachment 3,833 

Grand Rapids Labor Force Attachment 3,012 

Riverside Labor Force Attachment 6,726 

Lacked high school diploma or basic skills 3,125 

Portland 5,547 

Atlanta Human Capital Development 3,881 

Grand Rapids Human Capital Development 2,997 

Riverside Human Capital Development 3,135 

Columbus Integrated 4,672 

Columbus Traditional 4,729 

Detroit 4 459 

Oklahoma City 8,677 



Ever employed in year lor 2 (%) 



66.1 


61.6 


4.5 *** 


7.3 


77.7 


70.1 


7.6 *** 


10.9 


60.2 


45.0 


15.1 *** 


33.5 


55.5 


38.9 


16.6 *** 


42.7 


72.1 


60.9 


1 1.2 *** 


18.4 


64.4 


61.6 


2.8 ** 


4.6 


75.4 


70.1 


5.3 *** 


7.6 


48.2 


38.9 


9.3 *** 


23.9 


73.9 


72.2 


1.7 


2.3 


73.5 


72.2 


1.3 


1.7 


62.3 


58.2 


4 j *** 


7.0 


64.1 


65.0 


-0.9 


-1.4 



Average total earnings in years 1 and 2 f$) 



5,820 


5,006 


813 *** 


16.2 


5,674 


4,639 


1,035 *** 


22.3 


5,488 


4,213 


1,276 *** 


30.3 


4,124 


3,133 


992 *** 


31.7 


7,133 


5,291 


1,842 *** 


34.8 


5,502 


5,006 


496 ** 


9.9 


5,219 


4,639 


580 ** 


12.5 


3,450 


3,133 


317 


10.1 


7,565 


6,892 


673 ** 


9.8 


7,569 


6,892 


677 *** 


9.8 


4,369 


4,001 


367 * 


9.2 


3,518 


3,514 


5 


0.1 
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Exhibit ES-5 (continued) 



Site and Program 


Sample 

Size 


Program 

Group 


Control 

Group 


Ditterence 

(Impact) 


Percentage 
Change (%) 






Average total AFDC payments 








received in years 1 and 2 ($) 




Atlanta Labor Force Attachment 


3,833 


4,553 


4,922 


-368.6 *** 


-7.5 


Grand Rapids Labor Force Attachment 


3,012 


5,944 


7,347 


-1,403.7 *** 


-19.1 


Riverside Labor Force Attachment 


6,726 


8,292 


9,600 


-1,308.0 *** 


-13.6 


Lacked high school diploma or basic skills 


3,125 


8,894 


1 0,302 


-1,408.4 *** 


-13.7 


Portland 


5,547 


5,818 


7,014 


-1,196.3 *** 


-17.1 


Atlanta Human Capital Development 


3,881 


4,634 


4,922 


-287.5 *** 


-5.8 


Grand Rapids Human Capital Development 


2,997 


6,512 


7,347 


-835.1 *** 


-11.4 


Riverside Human Capital Development 


3,135 


9,253 


10,302 


-1,048.8 *** 


- 10.2 


Columbus Integrated 


4,672 


4,775 


5,469 


-693.7 *** 


-12.7 


Columbus Traditional 


4,729 


4,939 


5,469 


-529.8 *** 


-9.7 


Detroit 


4,459 


8,457 


8,615 


-157.5 0.0 


- 1.8 


Oklahoma City 


8,677 


3,391 


3,624 


-233.0 *** 


-6.4 






Ever received any AFDC payments 








in final quarter of year 2 




Atlanta Labor Force Attachment 


3,833 


61.3 


67.0 


-5.7 *** 


-8.5 


Grand Rapids Labor Force Attachment 


3,012 


53.5 


60.9 


7 4 *** 


- 12.1 


Riverside Labor Force Attachment 


6,726 


50.0 


56.4 


-6.4 *** 


- 1 1.3 


Lacked high school diploma or basic skills 


3,125 


54.2 


60.0 


-5.9 *** 


-9.8 


Portland 


5,547 


41.3 


53.0 


-117 *** 


- 22.1 


Atlanta Human Capital Development 


3,881 


63.6 


67.0 


-3.5 ** 


-5.1 


Grand Rapids Human Capital Development 


2,997 


54.3 


60.9 


-6.5 *** 


-10.7 


Riverside Human Capital Development 


3,135 


56.0 


60.0 


-4.1 ** 


- 6.8 


Columbus Integrated 


4,672 


47.1 


53.8 


- 6.8 *** 


-12.5 


Columbus Traditional 


4,729 


49.3 


53.8 


-4.6 *** 


-8.5 


Detroit 


4,459 


70.1 


73.7 


-3.6 *** 


-4.8 


Oklahoma City 


8,677 


38.4 


40.8 


-2.5 ** 


- 6.0 



SOURCES: MDRC calculations from unemployment insurance (UI) earnings records and AFDC records. 
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the Riverside GAIN program of the late 1980s (another employment-focused, varied first activity 
program). Earnings gains in the other employment-focused programs in’ the evaluation were 
moderate, ranging from $813 to $1,276 (Atlanta LFA and Riverside LF A, respectively). Earnings 
gains in the education-focused programs were smaller; statistically significant gains ranged from 
$367 to $677 (earnings impacts in Riverside HCD and Oklahoma were not statistically signifi- 
cant). Neither of the two low enforcement programs (Oklahoma City and Detroit) produced sub- 
stantial earnings increases. . . 
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Exhibit ES-6 



Impacts on , Two-Year Earnings 



2000 T 




11 



Employment-Focused Education- Focused 

Job Search First Varied First Activity • Education or Training First Education or Training First 

High Enforcement High Enforcement High Enforcement ' Low Enforcement 



SOURCE: MDRC calculations from unemployment insurance (UI) earnings records. 



• Over time, the employment and earnings gains diminished in most of the 
employment-focused programs, but increased in most of the education- 
focused programs. By the end of the two-year follow-up period some of 
the education-focused programs had “caught up” to the employment- 
focused programs. 




ES-18 

33 



The earnings gains in two of the three LFA programs (Grand Rapids and Riverside) di- 
minished over time, as increasing numbers of control group members began finding jobs on their 
own. In the last quarter of year 2 the three LFA programs raised employment levels by only 4 
percentage points and increased average earnings by about $100. (Exhibit ES-7 shows employ- 
ment levels over the follow-up period, averaged across programs within each approach.) 

In contrast, gains increased in most of the education-focused programs. By the last quar- 
ter of year 2, impacts on employment and earnings for five education-focused programs (Atlanta 
and Grand Rapids HCD, Columbus Integrated and Traditional, and Detroit) were similar to or 
slightly larger than impacts for the three LFA programs: employment gains ranged from 3 to 6 
percentage points and earnings gains ranged from $93 to $179. Overall, these results underscore 
the importance of tracking the effects of education-focused programs over a longer period than 
two years. 

Unlike the effects in other employment-focused programs, in Portland positive effects on 
employment and earnings increased over time: in the last quarter of follow-up the program group 
employment level was 1 1 percentage points higher than the control group level, and the program 
group earned on average $310 more. These impacts are far larger than those of any other pro- 
gram in the evaluation. 

• Portland’s program produced the largest, most consistent increases in 
employment stability and job quality during the follow-up period. 

Portland’s employment-focused, varied first activity program increased the proportion of 
people who worked all four quarters of year 2 by 8 percentage points and who earned $10,000 or 
more in year 2 by 6 percentage points. At the end of year 2 (as measured from survey responses) 
the program increased the percentage of people working at full-time jobs and at jobs that offered 
health coverage. It also increased average hourly pay for those working, but this finding, since it 
is based on a nonexperimental comparison (different types of individuals in the program and con- 
trol groups may have been working) is more speculative. The Riverside LFA program also in- 
creased full-time employment with health benefits and higher hourly earnings, but to a lesser ex- 
tent than the Portland program. Contrary to expectations, the education-focused programs 
increased job quality to only a small extent or not at all by the end of two years. 

D. Public Assistance Receipt and Payments 

• All programs reduced AFDC receipt to some degree. On average, de- 
creases for the employment-focused programs were larger, but decreases 
for some education-focused programs rivaled or exceeded decreases for 
some employment-focused programs. 

All programs lowered the proportion of welfare recipients who would have reached a 
two-year welfare time limit, had one been in effect. Control group members in all but one site 
received AFDC for an average of 16 to 20 months during the two-year follow-up period. (The 
exception was Oklahoma City, where the all-applicant sample averaged 12 months of receipt.) 
The programs reduced the average number of months of AFDC receipt by 0.48 to 2.41 months 
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Exhibit ES-7 

Quarterly Impacts on Employment Over Two Years, 
Averaged Across Sites by Approach 




SOURCE: MDRC calculations from unemployment insurance (UI) earnings records. 



(or 2 to 16 percent). Two employment-focused programs, Grand Rapids LFA and Portland, pro- 
duced the largest decreases (2.21 months and 2.41 months, respectively). Decreases for the edu- 
cation-focused programs ranged from 0.48 to 1.58 months. 

In the last quarter of follow-up between 41 percent (in Oklahoma City) and 74 percent (in 
Detroit) of control group members received an AFDC check. Portland produced the largest re- 
duction in the proportion of sample members receiving AFDC at this point (12 percentage 
points). Among the other programs, reductions in the proportion receiving AFDC at the end of 
year 2 ranged from 6 to 7 percentage points for the three LFA programs and from 3 to 7 percent- 
age points for the education-focused programs. 

• AH programs but one decreased average AFDC payments over the two- 
year follow-up period. 

Control group members received AFDC payments over the two years averaging between 
$3,624 and $10,302 (including those who left welfare during the two-year follow-up period). 
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Three employment-focused programs (Grand Rapids and Riverside LFA and Portland) and one 
education-focused program (Riverside HCD) reduced payments by more than $1,000 (represent- 
ing decreases of 10 to 19 percent, relative to payments to the control group). (See Exhibit ES-8.) 
Three other programs, all education-focused (Grand Rapids HCD and Columbus Integrated and 
Traditional), also reduced two-year welfare expenditures per program group member by 10 per- 
cent or more. Detroit’s program produced only a slight, not statistically significant, decrease in 
AFDC payments over the two years. 

o Most programs reduced Food Stamp receipt and expenditures during the 
follow-up period. 

Eight of the 1 1 programs decreased average Food Stamp expenditures over the two-year 
follow-up period and decreased the proportion of people who received Food Stamps in the last 
quarter of year 2. Decreases in two-year expenditures ranged from 2 to 13 percent and decreases 
in receipt at the end of follow-up ranged from 4 to 8 percentage points. One employment-focused 
program and two education-focused programs had no effect on Food Stamp receipt (Atlanta LFA 
and HCD and Oklahoma City). 

E. Employment and Welfare Status at the End of Two Years 

© In all programs a substantial proportion of enrollees were receiving 
AFDC at the end of the two-year follow-up period. 

Across all programs as many as 7 in 10 program group members (in Detroit) remained 
on welfare at the two-year mark. Even in programs that moved the largest proportion of sample 
members off welfare, at least 4 in 10 enrollees remained on welfare. This offers a caution to 
states striving to achieve very rapid self-sufficiency for virtually all welfare recipients. 

© Most programs increased the proportion of people who were working and 
not receiving AFDC at the end of the follow-up period. 

In the last quarter of year 2 between 13 and 27 percent of control group members were 
employed and receiving no AFDC payments. All programs but two (Riverside HCD and Okla- 
homa City) increased the proportion of people in this status. (See Exhibit ES-9.) Impacts were 
generally small, with two programs (Portland and Columbus Integrated) achieving moderate in- 
creases. The impacts, which ranged from 2 to 9 percentage points, were not associated with pro- 
gram approach. 

• Several programs representing both approaches slightly increased the 
rate at which individuals left welfare without a job. 

The proportion of control group members who were not employed and not receiving 
AFDC in the last quarter of year 2 ranged from 12 to 37 percent. All of the employment-focused 
programs and three of the seven education-focused programs increased the percentage of sample 
members in this status at the end of two years. (See Exhibit ES-9.) Increases were small in every 
program, ranging from 2 to 5 percentage points. The majority of people in this status reported 
having some other source of income and/or living with someone else who worked or who had 
another source of income. 
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Exhibit ES-8 



Impacts on Two-Year Welfare Payments 



Job Search First Varied First Activity Education or Training First Education or Training First 

High Enforcement High Enforcement High Enforcement Low Enforcement 



Employment Focused Education Focused 




-1600 

-1800 

-2000 



SOURCE: MDRC calculations from state and county AFDC records. 

F. Income and Poverty 

• Most programs had little or no effect on income. 

In the second year of follow-up control group members averaged between $5,596 (Okla- 
homa City) and $9,322 (Detroit) in combined income 'from earnings, AFDC, and Food Stamps. 
Few programs substantially altered these combined income levels; in general, reductions in 
AFDC, Food Stamps, and other benefits matched or exceeded earnings gains. However, in three 
programs — Grand Rapids and Riverside LFA (employment-focused) and Riverside HCD (edu- 
cation-focused) — combined income in the second year of follow-up was reduced by $230 to 
$571, or 3 to 7 percent. (See Exhibit ES-10.) In Portland (employment-focused) and Atlanta 
HCD (education-focused) combined income increased in the second year by $425 and $295, or 5 
and 4 percent, respectively. This combined income measure includes estimates of the EITC; 
when EITC estimates are not included, losses and gains are somewhat smaller, the Portland and 
Atlanta HCD gains are no longer statistically significant, and a small loss in the Grand Rapids 
LFA program becomes statistically significant. 
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Exhibit ES-9 

Impacts on Employment and Welfare Status at the End of Two Years 



Site and Program Employed Employed Not Employed Not Employed 

and off and on and on and off 

AFDC AFDC AFDC AFDC 



Atlanta Labor Force Attachment 


3.6 


0.8 


ns 


-6.5 


2.1 




Grand Rapids Labor Force Attachment 


2.8 


1.3 


ns 


-8.7 


4.6 




Riverside Labor Force Attachment 


2.3 


1.9 




-8.3 


4.1 . 




Portland 


9.3 


1.6 




-13.3 


2.5 




Atlanta Human Capital Development 


3.4 


2.7 




-6.2 


0.1 


ns 


Grand Rapids Human Capital Development 


4.1 


-0.1 


ns 


-6.4 


2.4 




Riverside Human Capital Development 


-0.7 ns 


2.6 




-6.7 


4.8 




Columbus Integrated 


6.6 


-1.6 


ns 


-5.2 


0.1 


ns 


Columbus Traditional 


4.6 


-1.1 


ns 


-3.5 


0.0 


ns 


Detroit 


2.9 


0.2 


ns 


-3.8 


0.7 


ns 


Oklahoma City 


-0.6 ns 


-0.5 


ns 


-2.0 


3.1 





SOURCE: MDRC calculations from administrative records. 
NOTE: Ns = not statistically significant. 



• Because income changes were minor, few programs lifted many families 
out of poverty. Some programs, however, had the effect of pushing a 
small proportion of families deeper into poverty. 

By design, the combined income from welfare and Food Stamp grants provides less than 
poverty-level income. Only by working can people hope to attain enough income to escape pov- 
erty. In the second year of follow-up between 1 1 and 26 percent of control group members had 
combined income from earnings, AFDC, Food Stamps, and estimated EITG receipt that equaled 
or exceeded the federal poverty level. Five programs increased the proportion of people living at 
or above poverty by a small amount. (See Exhibit ES-10.) Portland was the most successful, pro- 
ducing a 7.5 percentage point gain; impacts for other programs were small, ranging from 2 to 3 
percentage points. Program-control differences for most of the other seven programs were posi- 
tive but very small and not statistically significant. 
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Exhibit ES-10 



Impacts on Income and Poverty Status in Year 2 



Site and Program 


Combined 

Income 

($) 


At or Above 
Poverty Level 
(%) 


Below 50% of 
Poverty Level 
(%) 


Atlanta Labor Force Attachment 


246 ns 


2.9 


1.7 ns 


Grand Rapids Labor Force Attachment 


-230 ns 


1.5 ns 


4.7 


Riverside Labor Force Attachment 


-283 


2.4 


4.9 


Portland 


425 


7.5 


2.1 ns 


Atlanta Human Capital Development 


295 


2.8 


1 .9 ns 


Grand Rapids Human Capital Development 


-54 ns 


0.0 ns 


3.1 


Riverside Human Capital Development 


-571 


0.6 ns 


6.1 


Columbus Integrated 


47 ns 


1.5 ns 


2.5 


Columbus Traditional 


91 ns 


1.2 ns 


2.1 ns 


Detroit 


166 ns 


2.5 


-0.1 ns 


Oklahoma City 


-153 ns 


-0.1 ns 


2.4 



SOURCE: MDRC calculations from administrative records. 



NOTE: Ns = not statistically significant. 

All impacts include EITC estimates. 



In the second year of follow-up between 19 and 48 percent of control group members had 
combined income, including estimated EITC, totaling less than 50 percent of the poverty line. 
Six programs (both employment- and education- focused) slightly increased the proportion of 
sample members living below 50 percent of the poverty level; they led to increases of between 2 
and 6 percentage points in the proportion of individuals living deeply in poverty. 

G. Health Care Coverage and Child Care Expenses 

• Some programs that increased employment levels and decreased welfare 
receipt also decreased reported rates of health care coverage. 

At random assignment, almost all sample members in the evaluation had health coverage 
because they were receiving AFDC and were automatically covered under Medicaid. (In Okla- 
homa City, applicants for assistance whose eligibility was not yet determined were included in 
the sample, so initial coverage rates there were lower.) Over time, coverage rates declined for 
both program and control group members, as some people left AFDC and did not replace their 
Medicaid coverage with coverage from employers or other sources. By the end of the follow-up 
period between 81 percent (Columbus) and 88 percent (Detroit) of control group members re- 
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ported having health coverage for themselves and their children. (This range covers all sites ex- 
cept Oklahoma, where 68 percent reported coverage for themselves and their children.) 

Two employment-focused programs (Riverside LFA and Portland) and one education- 
focused program (Columbus Integrated) that increased employment and decreased welfare re- 
ceipt at the end of follow-up period also lowered health care coverage levels by 4 to 7 percentage 
points. (Impacts in Portland were not statistically significant, but were just beyond the . 1 level of 
statistical significance used as the standard throughout this report.) Although many program 
group members who left AFDC (and automatic Medicaid coverage) found jobs that provided 
health insurance, received Transitional Medicaid benefits, or obtained alternative sources of cov- 
erage, others were not able to replace the coverage they had under Medicaid. Some of these re- 
spondents never received Transitional Medicaid, and others had exhausted or had not restarted 
their benefits at the end of the two-year follow-up period. 

Program group members in Oklahoma City reported even larger decreases in coverage — 
11 percentage points. This program decreased welfare receipt and appears to have increased 
short-term employment — in jobs not reported to the states’ unemployment insurance system — 
that did not provide health insurance, especially for sample members’ children. The other seven 
programs in the evaluation did not affect health coverage rates for respondents or children. 

• Some programs increased child care use while employed and out-of- 
pocket child care expenditures, an increase due to greater child care use 
among those who found jobs as well as an overall increase in employment 
levels. 

Between 29 and 44 percent of all control group members (including those who never 
worked) used child care while employed at some point during the two-year follow-up period. 
Seven programs — the four employment-focused programs and three of the seven education- 
focused programs — produced moderate to large increases in child care use while employed, 
ranging from 4 to 13 percentage points. Impacts on paid child care use, that is, care paid for by 
either the sample member, the welfare department, the father of the child(ren), or the sample 
member’s employer, were found in nine programs and were similar in magnitude. 

The increases in child care use and in paid care use while employed are not entirely ex- 
plained by the programs’ impacts on employment; in many programs, of those who worked dur- 
ing the follow-up period, a greater proportion of program group members than control group 
members used child care (or paid care) as well. The likely explanation for this finding is that em- 
ployed program group members required or preferred more stable child care arrangements than 
employed control group members. This could be partly due to differences in the characteristics of 
the jobs acquired by program and control group members (for example, program group mem- 
bers’ jobs were more likely to be full time). It is also possible that program group members 
heeded the messages they were given by their caseworkers — messages probably delivered more 
frequently to program than control group members — concerning the importance of obtaining 
paid, stable child care. 

Relatively few program and control group members used transitional child care benefits. 
Five programs increased the use of such benefits, but these effects were large only in Atlanta 
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LFA and Portland, where increases of 7 and 1 1 percentage points in the receipt of these benefits, 
respectively, were found. 

H. Well-Being of Children 

• Some of the welfare-to-work programs affected children, although the ef- 
fects were not large or consistent across outcome measures or programs. 
Notably, the found effects on children were both positive and negative. 

The NEWWS Evaluation is one of the first random assignment evaluations of mandatory 
welfare-to-work programs to examine programs’ effects on the well-being of children. The chil- 
dren of sample members in the evaluation were often quite young. As noted earlier, in three of 
the sites women with children as young as age 3 were required to participate in welfare-to-work 
programs; in the other four sites the mandate was extended to include women with children as 
young as age 1. Because many of the child outcome measures used in the evaluation pertained 
only to children of school age, however, the child impacts discussed here are primarily for the 
subgroup of sample members who had no children under age 6. 

Control group members in the seven sites had, on average, two to three children. Across 
the sites an average of one-quarter of the control group members in the subgroup with no chil- 
dren under age 6 reported that at least one of their children had been suspended from school at 
some point during the two-year follow-up period. A smaller share of control group members — 8 
percent to 23 percent, depending on the site — reported having a child who had repeated a grade 
in school during the follow-up period. A relatively small proportion of all control group members 
— less than 8 percent in any site — reported that a child had been removed from their care dur- 
ing the two-year follow-up period. 

On measures of children’s behavioral adjustment, such as suspension from school, eight 
of the programs produced at least one statistically significant effect on children among the sub- 
group of families with no children under age 6. (See Exhibit ES- 11.) Three programs decreased 
the incidence of at least one behavioral problem, and five programs increased the frequency of at 
least one. Only two programs, however, had an effect on more than one behavioral adjustment 
measure. Fewer program effects were found on children’s progress in school, such as grade repe- 
tition, than on behavioral problems. Only two programs had any effects in this area, but, notably, 
these effects were favorable. Effects on children’s health and safety were also rare. Only two 
programs had any effect on children being removed from their mother’s care (small increases in 
the incidence of this event) and no programs affected the likelihood of children being taken to the 
hospital because of an accident, injury, or poisoning. 

• No explanations are clearly evident regarding the mechanisms through 
which some of the programs affected children. 

Program-specific differences in employment/education focus, sanctioning practices, and 
impacts on adult educational attainment, employment, and household composition could not be 
clearly linked to the programs’ effects on children. It could be that reductions in income played a 
role: some evidence suggests that those few programs that raised earnings levels, but reduced 
welfare and Food Stamps even more, resulted in adverse effects on children. In addition, child 
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care policies may have made a difference in the programs’ effects on children. Finally, the envi- 
ronments in which the programs operated (for example, their labor markets) may have been im- 
portant. Further research is needed to decide if or how these factors mediate the effects of wel- 
fare-to-work programs on children. 3 

I. Key Subgroups of Welfare Recipients 

• Interestingly, employment-focused programs were more likely than edu- 
cation-focused programs to achieve employment and earnings gains over 
the two-year follow-up period for those who entered the study without a 
high school diploma or GED certificate, but the difference in impacts 
narrowed by the end of the second year. 

Many programs produced employment and earnings gains for both those with and those 
without a high school diploma or GED at random assignment. (See Exhibit ES-12.) Among non- 
graduates all of the employment-focused programs boosted two-year employment levels — by 
more than 10 percentage points in Riverside LFA and Portland — and increased average earn- 
ings per program group member in year 2. In contrast, only three of the seven education-focused 
programs increased employment levels over two years, and only two programs increased year 2 
average earnings. 

At the end of the follow-up, however, one education-focused program (Columbus Inte- 
grated) was achieving the largest earnings gains of any program for nongraduates, and two others 
(Grand Rapids HCD and Columbus Traditional) attained larger earnings and/or employment im- 
pacts than two of the employment-focused programs (Atlanta and Riverside LFA). These results 
suggest that additional follow-up will be necessary to determine which kind of program approach 
is more effective for nongraduates in the long run. 

• Several programs produced moderate to large employment and earnings 
gains for the “most disadvantaged” sample members. 

Between 5 percent (Oklahoma City) and 28 percent (Riverside HCD) of the sample 
members in each site were welfare recipients who at study entry did not have a high school di- 
ploma or GED, had not worked in the prior year, and had received AFDC cumulatively for two 
years or more. Only a small proportion of control group members in this most disadvantaged 
subgroup became employed on their own during the two-year follow-up period (less than half in 
any site). 



3 The Child Outcomes Study, conducted by Child Trends as part of the NEWWS Evaluation, also examines the 
effects of welfare-to-work programs on the children of LFA, HCD, and control group members in Atlanta, Grand 
Rapids, and Riverside. This study uses a more comprehensive set of data about young children’s development, but 
only for children aged 3 to 5 at random assignment. See Sharon M. McGroder et al.. Impacts on Young Children 
and Their Families Two Years After Enrollment: Findings from the Child Outcomes Study (Washington, D.C.: U.S. 
Department of Health and Human Services and U.S. Department of Education, 2000). For a synthesis of the child 
research conducted thus far as part of the NEWWS Evaluation, see Gayle Hamilton, Do Mandatory Welfare-to- 
Work Programs Affect the Well-Being of Children? A Synthesis of Child Research Conducted as Part of the Na- 
tional Evaluation of Welfare-to-Work Strategies (Washington, D.C.: U.S. Department of Health and Human Ser- 
vices and U.S. Department of Education, 2000). 
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Five programs (Grand Rapids and Riverside LFA and HCD and Portland) increased em- 
ployment and earnings for the most disadvantaged subgroup. (See Exhibit ES-13.) Each of these 
programs increased the proportion who worked for pay during the follow-up period by more than 
10 percentage points. Gains in year 2 earnings were moderate ($800 or more) in two employ- 
ment-focused programs (Grand Rapids LFA and Portland) and smaller (between $605 and $667) 
in the three other programs. These programs and two others (Detroit and Columbus Integrated) 
also reduced the amount of time that these most disadvantaged individuals received AFDC dur- 
ing the two-year follow-up period. 

• Overall, both program approaches were less successful in helping people 
who had worked in the year before program entry, that is, a less disad- 
vantaged subgroup of the caseload. 

Between 22 percent (Riverside HCD) and 55 percent (Oklahoma City) of sample mem- 
bers in each site had worked for pay during the year prior to random assignment; 63 to 89 percent 
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Exhibit ES-12 



Year 2 Earnings Impacts 

for Subgroups Based on Educational Attainment at Study Entry 



Site and Program 


With High School 
Diploma or GED 
($) 


Without 
High School 
Diploma or GED 
($) 


Atlanta Labor Force Attachment 


483 


427 


Grand Rapids Labor Force Attachment 


352 ns 


728 


Riverside Labor Force Attachment 


795 


375 


Portland 


1,371 


881 


Atlanta Human Capital Development 


439 


276 ns 


Grand Rapids Human Capital Development 


574 


312 ns 


Riverside Human Capital Development 


n/a 


121 ns 


Columbus Integrated 


383 ns 


779 


Columbus Traditional 


513 


412 


Detroit 


311 ns 


279 ns 


Oklahoma City 


1 ns 


-25 ns 



SOURCE: MDRC calculations from administrative records. 
NOTE: NS = not statistically significant. 




45 



ES-29 



of control group members with this characteristic, depending on the site, became employed at 
some point during the two-year follow-up period. 

Only two employment-focused programs (Grand Rapids LFA and Portland) and one edu- 
cation-focused program (Grand Rapids HCD) increased both employment and earnings beyond 
what would have happened in the absence of the programs, for these sample members. Given the 
large proportion of control group members in this subgroup with employment and earnings in the 
two-year follow-up, impacts for this subgroup, when expressed as a percentage change, were 
rather small. 

V. Conclusions 

This evaluation, which used a random assignment experiment, provides solid information 
about the effectiveness of various types of welfare-to-work program approaches. Its unusually 
strong research design isolates the effects of the programs themselves; the results reported above 

National Evaluation of Welfare-to-Work Strategies 
Exhibit ES-1 3 



Year 2 Earnings Impacts for Subgroups 
Based on Level of Disadvantage at Study Entry 



Site and Program 


Most 

Disadvantaged 

($) 


Worked in 
Prior Year 
($) 


Atlanta Labor Force Attachment 


380 


191 ns 


Grand Rapids Labor Force Attachment 


800 ns 


682 


Riverside Labor Force Attachment 


613 


387 ns 


Portland 


838 


631 


Atlanta Human Capital Development 


12 ns 


23 ns 


Grand Rapids Human Capital Development 


667 


515 


Riverside Human Capital Development 


605 


-122 ns 


Columbus Integrated 


448 


613 


Columbus Traditional 


279 ns 


340 ns 


Detroit 


191 ns 


586 


Oklahoma City 


-90 ns 


8 ns 



SOURCE: MDRC calculations from administrative records. 



NOTE: Ns = not statistically significant. 



thus can be confidently attributed to the programs operated in the seven sites and not to im- 
provements in the sites’ labor markets, population changes, or other policy reforms. 
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The report’s findings, in conjunction with those of previous studies, suggest that strongly 
employment-focused programs that offer a variety of employment services are more effective 
than programs that offer primarily job search or education and training. Portland’s employment- 
focused, varied first activity program stands out as unusually successful among the 1 1 programs 
in this evaluation. The Riverside GAIN (Greater Avenues for Independence) program of the late 
1980s, often considered the benchmark for other welfare-to-work programs, was also an em- 
ployment-focused, varied first activity program. Both Portland and Riverside GAIN substantially 
increased employment levels, produced the largest earnings gains ever found for mandatory wel- 
fare-to-work programs, and had large impacts on welfare receipt. Both were successful for a wide 
range of subgroups, including the more disadvantaged members of the caseload. Operationally, 
the programs stressed the importance of finding jobs and enforced program participation re- 
quirements, but they offered many different services, including job search (along with job devel- 
opment), short-term education, and, in Portland, training. In both programs people considered not 
ready to enter the labor market were first assigned to basic education or, in Portland, to training 
or life skills classes. Although the 1996 welfare law encourages an employment focus, the avail- 
able research findings indicate that states can augment the success of their programs by offering 
education and training as well as job search. 

The report also illustrates, however, the limitations of even high-performing welfare-to- 
work programs: Although all of the programs in this evaluation had some positive effects, they 
generally did not produce large changes in people’s lives during the follow-up period. For exam- 
ple, the programs helped a substantial number of individuals replace income from AFDC and 
Food Stamps with income from jobs, but had not, as of two years, lifted many families out of 
poverty. (Additional. years of follow-up may show income gains, partly because of the increase in 
the value of the EITC in recent years.) Also, although all programs reduced welfare dependency 
to some degree, many people were still on welfare at the end of the two-year follow-up period 
(between 38 and 70 percent of those subject to the programs, depending on the site). 

Proponents of welfare time limits contend that the impending assistance cutoff will spur 
people into the labor market and promote self-sufficiency. The programs in this evaluation, 
which are similar to many programs being run under the new welfare law, operated without such 
a welfare time limit. (In addition, these programs did not try to meet the new law’s participation 
goals, impose full-family financial sanctions, or put in place the generous financial work incen- 
tives of many current programs. They also did not have available to them the recent and substan- 
tial increases in federal funding for child care or expanded eligibility for health insurance through 
Medicaid and the State Children’s Health Insurance Program.) Future research will indicate 
whether programs run in conjunction with time limits or other recent welfare policy changes will 
be considerably more successful than the programs previously operated. The present study does 
suggest, however, that strategies are needed to enable newly employed individuals to keep work- 
ing and to help them raise their earnings. Even in the very successful Portland program, less than 
one-third of all program group members worked in all four quarters of the second year of follow- 
up; less than one fifth of the total sample earned at least $10,000 in that same year. Future pro- 
grams will need to produce more sustained employment impacts and much bigger earnings im- 
pacts than those produced by any pre-TANF program that has been studied so far if large num- 
bers of people are to find employment that can adequately support their children before reaching 
a welfare time limit. 
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Chapter 1 

Introduction 



For the past 30 years, federal and state policymakers have been looking for new and bet- 
ter ways to help welfare recipients go to work. Beginning in the late 1 960s, in response to dissat- 
isfaction with the Aid to Families with Dependent Children (AFDC) program, Congress began to 
reshape it, creating a program to encourage welfare recipients to get jobs. In 1988 the Family 
Support Act (FSA) established a system of mutual obligation within the AFDC entitlement struc- 
ture, under which government was to provide education, employment, and support services to 
AFDC recipients who were, in turn, required to participate in the Job Opportunities and Basic 
Skills Training (JOBS) program. The most recent federal reform effort, the 1996 Personal Re- 
sponsibility and Work Opportunity Reconciliation Act (PRWORA), replaced AFDC with a block 
grant program, Temporary Assistance for Needy Families (TANF); limited most families to five 
years of federal TANF assistance (with some states having shorter welfare time limits); and cre- 
ated financial incentives for states to run mandatory, work-focused welfare-to-work programs. 
PRWORA gives states more flexibility than before in designing their programs (which has en- 
couraged, for example, more states to implement generous financial work incentives) and more 
responsibility for moving the nation’s poor into the labor market. 

This report contains the two-year results (within the overall period 1991-96) from an 
evaluation of 1 1 welfare-to-work programs — the National Evaluation of Welfare-to-Work 
Strategies (NEWWS) — begun under the FSA. The evaluation is being conducted by the Man- 
power Demonstration Research Corporation (MDRC) under contract to the U.S. Department of 
Health and Human Services (HHS), with support from the U.S. Department of Education. Child 
Trends, as a subcontractor, is conducting the analyses of outcomes for young children (the Child 
Outcomes Study). The evaluation includes programs in seven sites across the country: Atlanta, 
Georgia (Fulton County); Grand Rapids, Michigan (Kent County); Riverside, California (River- 
side County); Columbus, Ohio (Franklin County); Detroit, Michigan (Wayne County); Okla- 
homa City, Oklahoma (Oklahoma, Cleveland, and Pottawatomie counties); and Portland, Oregon 
(Multnomah and Washington counties). 1 The strong random assignment design of the evaluation 
provides solid information on the types of welfare-to-work program approaches that can move 
substantial numbers of people into work and off welfare without adversely affecting their fami- 
lies’ or children’s well-being. 

Two other reports should be viewed as “companion” documents to this report: Impacts on 
Young Children and Their Families Two Years After Enrollment: Findings from the Child Out- 
comes Study (McGroder et al., 2000), and Do Mandatory Welfare-to-Work Programs Affect the 
Well-Being of Children? A Synthesis of Child Research Conducted as Part of the National 
Evaluation of Welfare-to-Work Strategies (Hamilton, 2000). 



'The programs and individuals studied in this evaluation are drawn from the entire county (or counties) men- 
tioned in parentheses after the city name; for ease of reference, in this report the sites will be referred to by the name 
of their corresponding urban area. 
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I. A Framework for Understanding the Programs’ Results 

The FSA gave program administrators a great deal of flexibility in designing the 1 1 pro- 
grams studied in the National Evaluation of Welfare-to-Work Strategies. That flexibility, com- 
bined with local economic, political, and funding environments, resulted in 11 programs that 
vary on several dimensions of implementation. This report focuses on two of those dimensions: 
the self-sufficiency approach used and the level of participation mandate enforcement. 

While the overarching goal of programs run over the past 30 years — to foster the self- 
sufficiency of recipients through increased employment and decreased welfare receipt — has not 
changed, there has been disagreement on how best to move individuals from welfare to work. 
One strategy emphasizes quick employment, reflecting the belief that individuals can best build 
their employability, and eventually achieve self-sufficiency, through actual work, even if their 
initial jobs are minimum wage and without fringe benefits. The other strategy emphasizes skill- 
building, particularly in the education area, reflecting the view that individuals should first invest 
in education or training, to enable them to obtain higher-wage, longer-lasting jobs with health 
insurance coverage. Most programs have blended the two strategies and emphasized elements of 
both. Past research has shown that a program’s location on the theoretical continuum between 
these two strategies, and the mix of services it provides to enrollees, can have a distinct effect on 
the patterns and magnitude of program impacts measured in the short and long term. 2 

The programs in this report have been divided into those that use an employment-focused 
approach and those that use an education-focused approach to promote self-sufficiency. The re- 
port builds on and expands an earlier examination of three sites in the evaluation (Atlanta, Grand 
Rapids, and Riverside) that simultaneously implemented a Labor Force Attachment (LFA) pro- 
gram and a Human Capital Development (HCD) program, versions of employment-focused and 
education-focused programs that magnified the differences between the two types of approaches. 
These six programs in these three sites provide the best test of the relative effectiveness of the 
two approaches. 3 The final site, Columbus, was also asked to implement two different programs 
in a head-to-head test. One program used an “integrated case management” staffing structure, in 
which one worker assumes responsibility for both eligibility and employment and training for her 
clients. The other program used a “traditional case management” staffing structure, in which 
separate workers handle the eligibility and employment and training duties. These programs are 
called the Columbus Integrated and Traditional programs. Program administrators in the other 
four sites chose which self-sufficiency approach to implement based on their own goals. Taken 
together, four programs (Atlanta LFA, Grand Rapids LFA, Riverside LFA, and Portland) were 
employment-focused; the remaining seven were education-focused. 

In the LFA versions of the employment-focused program, almost all enrollees were first 
assigned to job search. In Portland, the other employment-focused program, many, but not all, 
individuals were assigned to job search as a first activity. Some individuals, usually those who 
were determined to have more barriers to work than other members of the caseload, were first 
assigned to education or training activities. In the three HCD education-focused programs, as 
well as in the four other education-focused programs, almost all individuals were first assigned to 
either education or occupational skills training activities. 



2 See Friedlander and Burtless, 1995; Bloom, 1997; and Riccio, Friedlander, and Freedman, 1994. 

3 Hamilton et al., 1997, presents two-year results for the six programs; future reports from this evaluation will 
document how each fared in the long term. 
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Past research also suggests that the degree to which a program enforces a participation 
mandate for the welfare caseload is a determinant of whether a program can have an effect. 4 High 
or low enforcement of the mandate is a product of three factors: how wide a cross section of the 
welfare caseload is enrolled in a program; how closely a program monitors individuals’ partici- 
pation; and how swiftly and consistently a program imposes financial sanctions, that is, reduc- 
tions in monthly welfare grants, on those who do not participate. Nine of the NEWWS programs 
were high enforcement programs; Detroit and Oklahoma City were not, mostly because of lim- 
ited program and staff resources. 

Table 1.1 categorizes the 11 NEWWS programs according to their self-sufficiency ap- 
proach and level of enforcement of the participation mandate. Chapter 3 discusses in greater de- 
tail these dimensions of the programs, as well as others that may have affected program impacts. 



National Evaluation of Welfare-to-Work Strategies 
Table 1.1 

Categorizing NEWWS Programs, by Approach, First Activity, and Enforcement Level 



Employment-focused approach 


Education-focused approach 


Job search first Varied first activity 


Education or Training 


High Enforcement High Enforcement 


High Enforcement Low Enforcement 


Atlanta LFA Portland 

Grand Rapids LFA 
Riverside LFA 


Atlanta HCD Detroit 

Grand Rapids HCD Oklahoma City 

Riverside HCD 
Columbus Integrated 
Columbus Traditional 



II. Research Questions and Design 

Within the above categorization scheme, the report analyzes the programs’ effects for 
single-parent welfare recipients, focusing on results for the two years after individuals entered 
the programs, an important period in which to gauge whether programs moved recipients from 
welfare to work. Presently, many states and localities terminate welfare eligibility after two 
years. In addition, prior research has shown that many individuals on welfare for at least two 
years will likely remain on the rolls for a considerably longer time. 5 Under TANF, these 
individuals would be in jeopardy of reaching their five-year limit on federal funding for welfare 
benefits. Consequently, the two-year results for these 1 1 programs will serve as a benchmark for 
the next generation of welfare initiatives. 

Specifically, the report addresses the following questions: 

• Which welfare-to-work program approaches were most successful in helping 
welfare recipients to receive the program services or attain the skills or 
credentials that could enhance their chances of finding employment? 



4 See Bloom, 1997, p. 51; Kemple, Friedlander, and Fellerath, 1995, pp. 



1987, pp. vii-x. 

5 See Bane and Ellwood, 1983. 



ES-2 and ES-3; and Friedlander et al., 
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• Which approaches were most successful in helping welfare recipients to find 
paid work and leave welfare within the two-year follow-up period, and to 
remain off welfare? Did any approach help individuals to get a “good” job, 
that is, a full-time job with health benefits? 

• Which approaches were most successful in increasing welfare recipients’ 
income and helping them move out of poverty? 

• Did any approaches, as a result of the services provided to or the mandates 
imposed upon parents, positively or negatively affect the well-being of their 
children? 

• Which approaches were most successful in achieving self-sufficiency for those 
who were at high risk for long stays on welfare? 

The NEWWS Evaluation uses an unusually strong research design, a random assignment 
experiment, to estimate program effects. In each site individuals who were required to participate 
in the program were assigned, by chance, to either a program group, which had access to em- 
ployment and training services and whose members were required to participate in the program 
or risk a reduction in their monthly AFDC grant, or a control group, which received no services 
through the program but could seek out such services from the community. This random assign- 
ment design assures that there are no systematic differences between the background characteris- 
tics of people in the program and control groups when they enter the study. Thus, any subsequent 
differences in outcomes between the groups can be attributed with confidence to the effects of 
the program. These differences, called impacts, are the primary focus of this report. 

Although this design assures that the impact estimates of each program are extremely re- 
liable, there are limitations to making cross-site comparisons of program effects. Local condi- 
tions, including labor markets, prevailing wages,' welfare grant levels, political environments, 
program funding levels, and staff administration, can all have an effect on the magnitude of im- 
pact estimates. For this reason impact differences among the 1 1 employment- and education- 
focused programs in this report are suggestive of the relative effectiveness of either approach in 
the short term. More definitive judgments on the relative effectiveness of the two approaches will 
come from the results from the three sites in this evaluation that are testing versions of the two 
approaches side by side. Furthermore, two years is not enough time in which to fully assess the 
effectiveness of either approach. Theoretically, only the results in later years of the follow-up 
period are expected to show a “payback” in the labor market from investments that participants 
in education-focused programs made in education and training. Future NEWWS reports will 
document these programs’ results in the longer term (for up to a five-year period). 

The remainder of this chapter briefly describes the localities in which the programs oper- 
ated and concludes with a description of the contents of the report. 



III. Program Environments 

When planning this evaluation, HHS and MDRC sought to include sites that would dem- 
onstrate operation in a diverse range of conditions, though they would not represent all welfare- 
to-work programs in the country. As shown in Table 1.2, sites varied along several dimensions, 
such as geographic location, labor market, and welfare grant level. 6 



6 For a description of the site selection process, see Hamilton and Brock, 1994, Appendix A. 
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all sample members in the NEWWS Evaluation were excluded from them. 



To be included in the National Evaluation of Welfare-to-Work Strategies, sites needed 
large enough welfare caseloads to meet the sample size requirements of the research design. Ac- 
cordingly, all of the seven sites include urban areas. Detroit, with a population topping 2 million 
in 1990, is the largest urban area studied in the evaluation, and the only site to lose population 
(by 3 percent) between 1990 and 1995, roughly the time period covered in this report. 7 Riverside, 
with a population of over 1 million in 1990, experienced the most growth during this period, add- 
ing almost 18 percent to its population by 1995. Population growth in other sites ranged from 5 
to 10 percent. 8 

As population grew, so did labor markets. In four sites employment expanded signifi- 
cantly between 1991 and 1996: the employed labor force in Grand Rapids grew by 16 percent, in 
Atlanta and Portland by 15 percent each, and in Riverside by 12 percent. The other three sites 
experienced 5 to 9 percent gains. 

Rising employment, particularly in localities with rising population, does not necessarily 
indicate declining unemployment rates. Unemployment rates in all seven sites, however, de- 
creased over this period. Following national trends, in general, unemployment rates peaked in 
1992 and were lowest in 1996. At the end of the evaluation period unemployment rates in most 
sites were below the national average of 5.4 percent in 1996. Early in the evaluation period un- 
employment rates in Detroit and Riverside topped 10 percent. Although rates in both localities 
steadily declined, Riverside’s remained at 8 percent in 1996, significantly higher than the na- 
tional average. Throughout the evaluation period Columbus’s labor market was robust; its unem- 
ployment rate never exceeded 5 percent, even during the high point of the national recession. 

Because individuals in the program and control groups within each site were subject to 
the same labor market, the quality of the economy by itself should not affect impact estimates; 
program and control groups shared the same advantages of a tight labor market or disadvantages 
of a slack one. However, different economic environments can present new opportunities or chal- 
lenges for welfare-to-work programs. For example, in a good labor market programs focused on 
job development will have an easier time locating and directing their clients to jobs to which 
control group members would not have access. In a slack labor market programs may choose to 
encourage recipients to invest in skills or education. 

The size of AFDC caseloads varied with the size of sites’ populations, ranging from 
about 7,500 in Grand Rapids to almost 90,000 in Detroit in 1991, the beginning of this evalua- 
tion. In general, sites’ welfare and program caseloads grew in the early part of the evaluation pe- 
riod, peaked in 1993 or 1994, and declined to their 1991 levels or below by 1996. Although this 
information is not available in all sites, a small percentage of the entire caseload actually partici- 
pated in the sites’ welfare-to-work programs. The program caseloads presented in Table 1.2, 
which represent annual unduplicated counts of program participants, grew substantially over the 
evaluation period. 



7 The time period covered by this report varies for each sample member; an inclusive calendar period is June 
1991 through December 1996. 

8 Data presented in this chapter are for the entire county (or counties) from which each site draws its sample 
members. 
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